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Why  Simulation? 


* 


INTRODUCTION 


Only  a  short  time  ago  it  was  quite  difficult  to  obtain  funds  for  a  simulation 
project  outside  certain  areas  of  very  specialized  research.  Practical  managers 
who  really  expected  useful  answers  to  questions  concerning  operating  problems 
were  rarely  interested  in  Investing  money  in  simulation.  In  the  past  2  years  this 
reluctance  seems  to  have  diminished  considerably.  Simulation  is  now  looked  on 
in  many  areas  as  a  practicable  means  to  obtain  answers  to  questions  raised  by 
managers  running  a  business,  operating  a  government  agency,  or  conducting  a 
military  operation.  Simulation  has  joined  the  growing  number  of  “OK  words* 
used  in  our  profession.  Simulation  is  'in*  along  with  management- information 
systems,  real  time,  and  timesharing. 

In  many  respects  we  should  be  delighted  with  this  turn  of  events.  Simula¬ 
tion  is  generally  a  great  deal  of  fun.  It  is  very  much  like  being  given  a  giant 
erector  set  with  which  one  can  build  the  most  elegant  and  esoteric  toys  ever. 

Most  desirable  of  all— simulation  is  considered  to  be  a  form  of  research,  and 
therefore  the  practitioner  is  seldom  faced  with  embarrassing  questions  about 
the  real  value  of  his  expensive  project.  With  any  luck  at  all  the  simulation- 
project  manager  can  so  dazzle  the  customer  with  tales  of  the  elegance  of  the 
simulation  language  and  complexity  of  the  model  that  he  may  never  be  asked  if 
any  useful  product  has  been  obtained. 

Of  course  there  have  been  many  useful  and  productive  simulation  projects. 
Fortunately,  these  successes  have  occurred  with  a  high  enough  frequency  to 
maintain  our  faith  in  the  value  of  the  technique.  My  concern  is  that  there  seem 
to  be  altogether  too  many  simulation  models  being  built  that  are  unlikely  to 
produce  any  useful  results.  More  dangerous  still,  there  seem  to  be  many  sim¬ 
ulation  models  that  produce  apparently  useful,  but  unverif table  answers.  They 
must  be  either  ignored  or  accepted  on  faith. 

From  a  professional  point  of  view,  I  am  concerned  that  we  are  overselling 
simulation  as  a  technique.  From  a  practical  point  of  view,  I  am  concerned  that 
the  pendulum  may  well  swing  back  in  the  next  2  years  to  the  point  where  sim¬ 
ulation  is  regarded  as  an  untrustworthy  and  expensive  form  of  boondoggle. 
Simulation  is  an  extremely  valuable  tool  when  used  properly  to  attack  problems 
amenable  to  simulation,  but  a  most  wasteful  and  Inappropriate  tool  when  other 
techniques  are  available. 

Now  that  people  have  lost  their  skepticism,  the  next  problem  is  to  develop 
our  own  restraints  if  we  are  to  avoid  wasteful  and  misleading  simulation  projects. 
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DEVELOPMENT  OF  SIMULATION 


II  the  limitations  of  a  simulation  are  to  be  discussed,  first  we  must  define 
simulation.  The  literature  in  the  field  is  of  limited  usefulness  in  defining 
the  term  exactly.  The  more  recent  the  document,  the  more  likely  it  is  that  the 
author  will  quote  several  alternative  definitions,  reject  them  all,  and  proceed 
to  develop  his  own,  which  is  more  general  and  less  restricting  than  any  of  those 
given  (see  Naylor  et  al1)-  The  use  of  the  term  “simulation”  seems  to  outgrow 
even  the  broadest  definition.  I  will  not  attempt  a  precise  definition  but  as  an 
alternative  will  suggest  that  the  history  of  the  development  of  simulation  as 
it  is  known  today  may  be  a  less  rigorous  but  more  satisfying  way  to  describe 
simulation. 

The  origins  of  simulation  are  generally  traced  to  the  work  of  von  Neuman 
and  Ulman  in  the  late  1940’s.  They  coined  the  term  “Monte  Carlo  analysis,”  to 
describe  a  technique  whereby  essentially  deterministic  problems  too  expensive 
or  too  complex  to  solve  analytically  could  be  solved  by  treating  them  as  stochas¬ 
tic  problems.  The  genius  of  their  method  was  that  it  was  an  inversion  of  the 
usual  approach  to  stochastic  problems— that  of  treating  stochastic  problems  as 
if  they  were  deterministic,  in  order  to  solve  them  analytically.  The  Monte  Carlo 
approach  was,  then,  the  process  of  finding  a  stochastic  analog  to  the  determin¬ 
istic  problem  in  order  to  estimate  the  solution  through  simulation. 

Figure  1  is  a  greatly  simplified  example  of  Monte  Carlo  analysis.  The  area 
enclosed  by  the  outline  of  the  glass  could  be  calculated  through  integration  of 
the  function  that  represents  the  sides  of  the  glass,  but  such  Integration  might 
be  difficult  and  costly.  The  Monte  Carlo  approach  would  be  to  enclose  the  cross 
section  of  the  glass  by  a  rectangle  of  known  area.  An  appropriate  sample  of 
randomly  generated  points  is  then  plotted  in  such  a  way  that  all  points  within  the 
rectangle  have  an  equal  chance  of  being  selected.  The  area  within  the  glass  is 
then  estimated  by  multiplying  the  fraction  of  all  points  that  fall  within  the  glass 
by  the  area  of  the  rectangle.  This  example  is  a  relatively  simple  two-dimen¬ 
sional  case;  however,  the  basic  technique  can  be  applied  to  problems  of  signifi¬ 
cantly  greater  complexity  that  may  not  be  solvable  by  analytical  means. 

The  development  of  Monte  Carlo  analysis  occurred  concurrently  with  the 
development  of  the  high-speed  computer,  and  as  a  result  most  applications 
were  soon  being  conducted  with  the  use  of  a  computer.  The  techniques  so  de¬ 
veloped  were  then  applied  to  problems  that  were  basically  stochastic.  Then 
followed  a  rapid  application  of  the  technique  to  many  of  the  classic  stochastic 
problems  not  amenable  to  analytic  solution,  e.g.,  processes  involving  multiple- 
channel  queues.  A  great  deal  is  known  about  scheduling  and  relating  problems 
as  a  result  of  the  availability  of  these  techniques  and  the  digital  computer.  In 
the  process  a  body  of  knowledge  concerning  the  computer  modeling  of  real-world 
processes  was  developed  that  has  become  the  real  basis  of  computer  simulation 
as  it  is  known  today. 

Figure  2  indicates  a  simple  example  of  the  stochastic  simulation  of  a 
stochastic  process.  Here  the  problem  is  the  interaction  of  the  arrival  of  orders 
and  the  service  time  of  each  of  the  processes  that  lead  to  queues  forming  ahead 
of  some  processes  and  a  particular  distribution  of  total  order-processing  time. 
Here  the  rate  of  arrival  of  new  orders  is  known, as  is  the  distribution  of  service 
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Fig.  1— Monte  Carlo  Stochastic  Approximation  of  a  Deterministic  Volue 


Fig.  2— Stochastic  Simulation  of  a  Stochastic  Process 
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tim  3  for  each  01  die  processes.  The  basic  simulation  approach  is  to  generate 
arrivals  and  service  times  by  selecting  from  the  known  distributions  of  the 
times  through  the  use  of  random  numbers.  Thus  the  whole  procedure  can  be  de¬ 
scribed  as  the  development  of  a  distribution  for  the  total  process,  based  on  de¬ 
tailed  knowledge  of  the  behavior  of  each  of  the  components. 

A  key  point  not  always  recognized  is  that  the  simulation  of  large  queueing 
systems  began  after  a  well-developed  theory  of  the  behavior  of  the  single- channel 
queue  was  available.  Large  simulation  models  of  queueing  systems  were  then 
constructed  using  components  that  were  understood.  The  success  of  this  general 
type  of  model  has  led  some  to  attempt  the  modeling  of  large  systems  in  which 
little  is  known  of  the  behavior  of  the  individual  components. 

A  somewhat  later  development  was  the  use  of  computer  modeling  techniques 
to  study  business,  economic,  and  organizational  problems.  Clearly  these  pro¬ 
cesses  are  probabilistic  in  nature  and  not  deterministic.  Yet  some  of  the  fore¬ 
most  practitioners  chose  to  develop  deterministic  models  of  these  systems. 

For  example,  Jay  Forrester’s  industrial  dynamics2  approach  is  essentially  de¬ 
terministic.  Forrester  chose  to  study  the  dynamic  character  of  industrial  sys¬ 
tems  first  without  the  noise  created  by  random  events;  only  later  did  he  introduce 
stochastic  elements  to  observe  their  effect.  This  approach  is  prompted  by  the 
belief  that  the  probability  functions  of  such  processes  are  not  yet  known  in  suf¬ 
ficient  detail  to  support  a  useful  stochastic  model  and  further  that  there  is  much 
to  be  learned  about  the  dynamic  properties  of  these  processes  aside  from  their 
probabilistic  elements.  Thus  an  essentially  deterministic  approach  is  used  to 
study  a  probabilistic  process. 

Figure  3  is  a  simple  example  of  the  deterministic  approach  such  as  might 
be  used  in  an  industrial  dynamics  simulation.  Here  the  focus  is  on  the  feedback 


Fig.  3 — Th«  Deterministic  Simulation  of  a  Stochastic  Process 


problem  of  control.  The  controlling  manager  observes  the  level  of  inventories 
and,  acting  on  this  information,  regulates  the  flow  of  resources  into  the  manu¬ 
facturing  process.  The  emphasis  here  is  on  the  oscillations  of  the  system  that 
may  be  created  by  the  information  and  control  mechanism. 
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Although  this  review  has  certainly  not  definid  simulation  in  any  rigorous 
way,  simulation  models  can  now  be  characterized  in  two  ways:  first  by  de- 
scribingwhat  they  are  not.  Simulation  models  do  not  purport  to  find  the  best 
solution  to  any  problem;  they  are  not  optimizing  in  any  sense.  The  best  to  be 
said  about  any  simulation  model  is  that  it  describes  the  object  process,  using 
characteristics  that  are  important  to  the  results  the  model  builder  wishes  to 
study.  A  simulation  model  thus  serves  its  function  by  demonstrating  the  con¬ 
sequences  of  a  particular  set  of  inputs  and  decision  rules  applied  to  the  process. 
The  model  builder  is  free  to  change  any  of  these  in  the  hope  of  improving  the 
result,  but  improvement  is  by  no  means  guaranteed.  Thus,  if  optimization  or 
improvement  is  the  model  builder’s  goal,  that  improvement  must  be  obtained 
through  inference  from  the  model’s  behavior. 

Second,  we  can  say  what  simulation  is  like.  Simulation  has  many  of  the 
characteristics  of  a  laboratory  experiment;  it  certainly  can  be  controlled. 

Any  run  of  any  simulation  can  be  replicated  precisely,  or  any  element  changed 
to  exactly  the  degree  wished.  Since  the  digital  computer  is  a  deterministic 
mechanism,  the  conditions  of  the  simulation  can  actually  be  controlled  to  run 
more  accurately  than  the  conditions  of  the  laboratory  model.3 

Simulation,  like  laboratory  experiment,  is  also  an  abstraction  from  the 
real  world.  Laboratory  experiments  seem  at  first  to  have  an  advantage  in  that 
we  usually  separate  some  material,  an  actual  part  of  the  real  world,  and  place 
it  in  the  controlled  conditions  of  the  laboratory.  One  is  tempted,  therefore,  to 
think  that  the  laboratory  experimenter  has  no  problems  in  proving  the  validity 
of  the  abstraction,  but  clearly  this  is  not  the  case.  The  laboratory  experimenter 
must  be  concerned  that  in  his  abstraction  the  basic  condition  he  wishes  to  study 
has  not  been  altered.  He  must  be  concerned  that  he  is  faithfully  reproducing 
in  the  laboratory  the  conditions  that  are  important  to  the  process  he  wishes  to 
explore.  This,  of  course,  is  precisely  the  problem  that  the  simulation- model 
builder  must  consider. 

If  simulation  is  in  effect  an  experimental  process,  a  number  of  things  can 
be  inferred  from  the  experience  gained  in  the  laboratory  experiment.  One  is 
that  laboratory  procedure  is  carefully  followed  and  reported  in  such  experiments. 
The  simulation  analog  of  this  may  be  the  strict  control  and  reporting  of  the 
random-number  generators  used,  the  collection  and  tabulation  techniques  used 
in  gathering  the  basic  data,  the  sample  sizes  used,  and  other  related  techniques. 

An  important  point  here  is  that  in  simulation  there  is  not  yet  a  formalized 
set  of  control  techniques  that  ensures  the  accuracy  of  simulation  experiments. 
The  skill,  experience,  and  thoroughness  of  the  individual  experimenter  are  all- 
important.  There  is  no  cookbook  formula  that  permits  the  nonparticipant  to 
assure  himself  that  the  results  of  a  simulation  experiment  are  valid. 

Another  similarity  between  a  simulation  and  laboratory  experiment  is  the 
results  that  one  can  reasonably  expect.  The  simulation- model  builder  hopes 
that  he  will  be  surprised  by  the  results  of  his  model.  He  hopes  that  the  model 
will  behave  in  a  manner  that  he  would  not  have  predicted  and  yet  not  violate 
any  of  the  known  characteristics  or  relations  of  the  object  system.  For,  if 
surprising  results  occur,  the  model  builder  may  have  discovered  some  new 
and  interesting  property  of  the  object  system  not  apparent  before. 

Of  course,  many  surprises  turn  out  to  be  the  result  of  a  failure  in  the 
model  rather  than  a  legitimate  key  to  new  understanding.  The  separation  of 
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surprises  into  these  two  categories  is  one  of  the  most  difficult  problems  of 
simulation. 

A  second  result  that  one  can  hope  for  from  simulation  and  experimentation 
is  that  the  relative  merit  of  two  or  more  courses  of  action  can  be  tested.  Ac¬ 
knowledging  the  imperfections  of  the  models,  one  may  still  hope  that  they  will 
be  capable  of  distinguishing  a  better  set  of  decision  rules  or  a  more  favorable 
set  of  inputs.  This  implies  the  measurement  of  the  result  against  the  changes 
to  the  model  or  its  inputs.  Consequently  the  measurement  of  results  is  as 
vital  in  simulation  as  it  is  in  physical  experimentation. 


ADVANTAGES  OF  SIMULATION 

The  limitations  as  well  as  the  advantages  of  simulation  are  perhaps  most 
succinctly  stated  by  Teichroew  and  Lubin4  as  the  following: 

Simulation  problems  are  characterized  by  being  mathematically  intractable  and 
having  resisted  solution  by  analytic  methods.  The  problems  usually  involve  many 
variables,  many  parameters,  functions  which  are  not  well  behaved  mathematically  and 
random  variables.  Thus,  simulation  is  a  technique  of  last  resort.  Yet,  much  effort  is 
now  devoted  to  ‘computer  simulation’  because  it  is  a  technique  that  gives  answers  in 
spite  of  its  difficulties,  costs  and  time  required. 

One  advantage  of  simulation  then  is  that  problems  not  otherwise  feasibly 
solved  are  open  to  investigation.  1  would  submit  that  this  is  the  only  true  ad¬ 
vantage.  In  all  other  respects— cost,  elapsed  time  required  for  solution,  skill 
required  of  the  analyst,  and  accuracy  of  the  results— simulation  is  second  best 
if  a  reasonably  realistic  analytic  solution  is  available. 

There  are,  of  course,  cases  where  analytic  or  experimental  solution  is 
theoretically  possible  but  practically  difficult  or  infeasible.  In  the  vast  majority 
of  cases,  however,  if  an  analytic  solution  is  available  it  should  by  all  means 
be  used.  The  point  here  is  that  simulation’s  single  overwhelming  advantage  is 
that  it  is  often  the  only  feasible  approach  to  important  problems. 

This  idea  is  not  always  popular  with  people  in  the  simulation  field.  Other 
additional  advantages  are  often  cited.  One  of  the  proposed  advantages  is  that 
simulation  permits  more  complex,  more  elegant,  and  hence  more  realistic 
models.  This  is  certainly  true.  The  number  of  variables  and  parameters  may 
in  some  cases  be  larger  (by  an  order  of  magnitude)  than  currently  available 
optimizing  models.  Types  of  functions  not  feasible  in  analytic  models  can  be 
used  relatively  easily  in  simulation  models.  The  richness  and  variety  of  the 
modeling  language  available  to  simulation  is  certainly  greater  by  far  than  in 
available  optimizing  models. 

The  real  question  is,  “Does  the  complexity  pay  off?”  Elegance  for  its 
own  sake  is  not  very  valuable.  Complexity  pays  off  only  if  it  truly  adds  to  the 
realism  of  the  model  and  if  that  realism  is  necessary  to  a  useful  solution. 

Thus  complexity  is  not  an  advantage  per  se,  but  only  if  it  is  the  sole  recogniz 
able  path  to  solution. 

Another  advantage  sometimes  cited  is  that  simulation  can  be  used  even 
in  the  cases  where  the  process  is  not  fully  understood  and/or  the  data  are  not 
complete.  This  too  is  quite  true.  If  one  is  willing  to  make  the  necessary 
assumptions,  a  simulation  model  can  be  developed  for  virtually  anything  that 


8 


can  be  conceived.  In  some  cases  the  construction  of  a  simulation  model  may 
be  excellent  preparation  for  the  gathering  of  additional  data.  The  process  of 
building  and  exercising  a  simulation  model  has  a  remarkable  way  of  focusing 
one’s  attention  on  the  need  for  specific  data,  and  the  necessity  for  better  under¬ 
standing  of  particular  aspects  of  the  object  process.  But  then  isn’t  this  a 
claim  that  can  be  made  for  all  model  building? 

There  is  much  to  be  said  for  an  approach  to  problem  solving  that  uses 
simulation  to  test  the  consequences  of  data  and  hypotheses  already  at  hand 
and  to  focus  on  the  areas  in  which  additional  work  needs  to  be  done.  Two 
things  must  be  said  of  this  approach:  First,  other  modeling  methods  can  make 
the  same  claim;  hence,  if  simulation  is  chosen,  it  should  be  because  no  simpler 
or  cheaper  approach  may  be  used.  Second,  there  is  great  danger  in  this  approach 
in  that  the  first  answers  are  all  too  often  taken  as  the  answers.  Thus  simulation 
in  the  face  of  poor  data  and  weak  theoretical  foundation  is  an  advantage  only  if 
the  weakness  of  the  outputs  as  well  as  of  the  inputs  is  recognized.  Simulation 
is  not  a  machine  that  converts  weak  inputs  and  assumptions  to  strong  results 
and  conclusions. 

Finally,  it  is  sometimes  stated  that  one  of  the  advantages  of  simulation  is 
that  the  manager  himself  can  understand  simulation  models  without  the  intensive 
training  or  technical  background  generally  thought  necessary  for  the  use  of 
optimizing  models.  This  conclusion  is  subject  to  considerable  argument.  FirBt, 
let  us  recognize  that  those  citing  this  as  an  advantage  are  generally  talking  about 
operating  problems  and  nontechnical  managers.  It  is  true  that  one  can  generally 
explain  the  basic  approach  to  simulation  to  nontechnical  managers  more  readily 
than  one  can  explain,  e.g.,  linear  programming.  Further,  the  outputs  of  simula¬ 
tion  are  frequently  easier  to  understand  because  they  are  themselves  simula¬ 
tions  of  object-system  reports.  If  this  advantage  is  restated  to  read,  ‘The  re¬ 
sults  of  simulation  models  are  easier  to  sell  because  they  can  produce  results 
similar  to  those  produced  by  the  object  system,*  then  I  think  we  must  admit 
this  is  true. 

Simulation  can  be  described  as  the  computerization  of  experience.  That  is, 
in  the  study  of  operating  processes  we  build  our  model  based  on  the  practical 
knowledge  of  the  object  process  and  then,  by  running  it  repeatedly,  gain  “years* 
of  experience  in  a  relatively  short  time.  Thus,  although  the  manager  may  not 
understand  the  intricacies  of  stochastic  processes,  he  can  appreciate  the  basic 
trial-and-error  approach. 

It  seems  to  me  that  this  advantage  is  also  a  danger.  Simulation  models, 
no  matter  how  thin  the  data  or  how  limited  the  understanding  of  the  object  pro¬ 
cesses,  can  almost  always  come  up  with  answers.  If  the  answers  are  super¬ 
ficially  understood,  a  source  of  misinformation  may  be  created.  I  would  there¬ 
fore  suggest  that  this  advantage  is  also  a  trap.  It  might  be  better  if  the  results 
of  simulation  could  not  be  understood  without  extensive  training  and  in-depth 
understanding. 


MODELING  PROBLEMS  AND  SIMULATION 

It  is  not  always  popular  to  refer  to  the  problems  that  are  encountered 
in  any  endeavor.  We  prefer  to  discuss  interesting  situations  or  challenges. 


In  simulation  there  are  a  number  of  interesting  situations  that  one  should  re¬ 
cognize  before  plunging  into  the  world  of  simulation  modeling. 

One  of  the  first  to  be  recognized  is  the  fact  that  good  simulation  requires 
skilled  and  experienced  analysts.  The  analyst’s  first  and  perhaps  most  im¬ 
portant  job  is  to  select  the  properties  of  the  object  system  that  are  important 
to  the  process  being  studied.  This  implies  deep  understanding  of  the  process 
itself,  and  considerable  skill  in  modeling.  The  model  builder  thus  needs 
thorough  familiarity  with  the  object  process,  skill  in  working  with  the  modeling 
language,  and  skill  in  exercising  the  model. 

Few  simulation  models  are  constructed  in  this  fashion.  Their  functions 
are  more  commonly  separated.  One  individual  or  group  examines  the  object 
process  and  determines  the  properties  that  are  important;  a  second  individual 
or  group  then  maps  these  properties  using  the  simulation  language.  In  many 
cases  a  third  individual  or  group  then  conducts  experiments  with  the  model 
and  interprets  the  results.  There  would  seem  to  be  no  necessity  for  having  a 
single  person  perform  these  tasks;  however,  there  is  certainly  a  clear  need 
for  superior  communication  to  ensure  that  the  modeling  process  or  the  ex¬ 
periments  do  not  go  beyond  the  capability  of  the  conceptual  model. 

A  second  requirement  is  thorough  technical  knowledge  of  the  statistical 
processes  involved.  Experiments  with  simulation  models  should  be  as  care¬ 
fully  controlled  as  any  physical  experiment.  The  achievement  of  the  steady 
state  by  the  model,  measurement  of  the  behavior  of  the  model,  and  selection 
of  the  sample  size  to  be  used  are  all  tasks  that  require  considerable  technical 
capacity  beyond  that  which  is  normally  considered  necessary  for  programming. 

Finally,  considerable  skill  in  planning  experiments  and  interpreting  their 
results  is  needed.  In  its  present  state  of  development,  simulation  is  an  art, 
not  a  science.  Experience  and  intuitive  skill  play  a  very  important  part  in 
differentiating  between  genuine  new  information  of  the  object  process  and  the 
anomalies  of  the  model  itself. 

A  fundamental  problem  affecting  all  models,  but  particularly  simulation 
models,  is  the  difficulty  of  determining  the  model’s  validity.  This  problem 
has  not  been  solved  in  any  rigorous  sense  and  may  never  be  solved  rigorously. 
One  aspect  of  the  problem  is  illustrated  by  the  following  two  statements: 

(a)  Simulation  models  are  valuable  because  they  permit  observation  of 
processes  otherwise  difficult  or  impossible  to  observe  in  the  real  world. 

(b)  Simulation  models  are  valid  if  they  behave  as  the  object  system 
behaves. 

These  statements  are  of  course  simplified  and  perhaps  imprecise  but 
they  do  summarize  valid  points.  A  model,  by  definition,  will  not  behave  in  all 
respects  as  the  object  system  behaves.  One  of  the  important  characteristics 
of  a  model  is  that  it  permits  concentration  on  the  parts  of  the  object  process 
that  are  believed  to  be  important.  Yet  ultimately  the  findings  that  are  most 
important  to  us  are  not  directly  observable  in  the  object  system.  One  can 
always  deflate  a  simulation- model  builder  by  classifying  all  his  results  in 
two  categories,  (a)  those  results  obtained  by  observing  the  system  directly 
and  (b)  those  results  that  are  unsupported. 

The  validity  problem  is  summarized  in  Fig.  4.  A  is  the  objec  t  system, 
and  B  is  that  portion  of  the  object  system  directly  observable,  such  that  B  is 
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a  subset  of  4  .*  C  is  then  the  model.  Now  the  model  by  definition  does  not 
contain  all  the  characteristics  of  the  real  system.  Thus  AC' and  ABC' are 
not  empty.  In  addition,  A'C  will  not  be  empty,  i.e.,  the  model  will  contain 
anomalies  that  are  characteristic  of  the  model  and  not  of  the  object  system. 
AB'C  is  valuable  information.  These  are  the  characteristics  of  the  model 


Fig.  4 — The  Object  System  Validity  Problem 
A  =  object  system 

8  =  observable  portion  of  the  object  system 
8  C  A 
C  -  model 

A'C  =  anomaly  of  the  model 
A8'C  -  valuable  information 
ABC  =  validity  confirmation  space 

and  of  the  object  system  that  are  not  observable.  ABC  is  the  validity  confirma¬ 
tion  space.  That  is,  it  is  in  this  region  that  the  validity  of  the  model  may  be 
tested  by  comparing  its  performance  with  the  observed  performance  of  the 
object  system. 

Now  the  above  discussion  is  a  conceptual  one,  but  it  illustrates  the  fun¬ 
damental  problem  of  simulation;  i.e.,  as  each  experiment  is  conducted  on  the 
model  the  results  may  be  classified  into  two  categories.  One  category  consists 
of  the  region  ABC,  which  contains  the  points  in  which  the  observable  portion  of 
the  object  system  and  the  model  coincide.  This  is  the  region  on  which  our 
'Taims  for  the  validity  of  the  model  is  based. 

The  second  classification  consists  of  the  union  of  A’C  and  AB’C  (A'C  u 
AB'C)-  That  is,  the  results  may  consist  of  invalid  and/or  valid  results  of  the 
model.  Herein  lies  the  crux  of  the  validity  problem.  The  valid  and  invalid 
results  cannot  be  distinguished  without  resorting  to  information  outside  the 
simulation  process. 

The  usual  defense  against  attacks  on  the  validity  of  one’s  model  is  to 
point  to  the  great  detail  and  precision  of  the  model  itself.  It  is  implied  that 
greater  detail  and  complexity  ensures  greater  realism  and,  it  is  hoped,  greater 
validity.  In  short,  simulation-  model  builders  frequently  find  themselveB  driven 
to  larger  and  larger  models  of  unfathomable  complexity.  One  gets  the  impres¬ 
sion  that  the  prize  goes  to  the  most  complex  model  rather  than  the  one  with  the 
most  useful  results. 


*B  will  not  be  a  subset  of  A  unless  our  observations  of  the  system  are  completely 
accurate.  A'B  is  In  fact  the  region  of  inaccurate  observations. 
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This  complexity  leads  to  some  curious  situations.  It  is  not  uncommon  to 
discover  that  a  simulation  model  of  a  sequential  process  is  made  up  of  com* 
ponents  that  vary  widely  in  their  levels  of  detail.  One  component  may  be  built 
in  the  greatest  detail  with  all  the  elegance  and  complexity  that  its  builder  can 
devise;  its  outputs,  however,  may  be  digested  by  the  next  component  in  such  a 
manner  as  to  lose  the  detail  generated  in  the  previous  step. 

Model  complexity  is  in  itself  a  problem.  It  is  the  unexpected  result  that 
is  most  desired  in  model  building  and  experimentation.  When  that  unexpected 
result  is  found,  however,  a  reasonable  explanation  is  normally  sought,  i.e., 
an  attempt  is  made  to  explain  the  outcome  as  the  result  of  known  properties  of 
the  object  system.  The  complexity  of  models  works  against  us  in  this  situation. 
It  is  found  sometimes  that  our  model  is  so  complex  that  we  are  unable  to  trace 
the  outcome  to  the  originating  causes  within  the  object  system.  When  this 
situation  occurs,  the  model  builder  lias  a  difficult  choice.  He  must  redesign 
the  model,  search  elsewhere  for  confirmation,  or  plunge  into  conclusions  know¬ 
ing  they  may  be  based  on  characteristics  in  the  model  but  not  in  the  object 
system. 

In  our  earlier  definition  of  simulation,  we  characterized  simulation  as 
nonoptimizing.  It  hardly  seems  fair  now  to  state  this  as  a  limitation.  I  would 
submit,  however,  that  this  characteristic  of  simulation  is  an  operational  limi¬ 
tation  as  well  as  a  theoretical  limitation. 

In  actual  practice  we  really  do  not  expect  to  optimize  real-world  processes 
as  a  result  of  ‘‘optimizing1’  models.  Such  models  tend  to  oversimplify  the  real 
world,  and  they  can  rarely  be  put  directly  into  practice.  Optimization  is  an 
elusive  goal;  it  is  extremely  difficult  if  not  impossible  to  formulate  our  true 
objectives  in  precise  form,  let  alone  solve  for  them. 

Optimizing;  models  do,  however,  have  one  great  operational  advantage: 
we  know  when  the  stated  objective  has  been  accomplished.  This  is  not  to  say 
that  the  model  cannot  be  reformulated  or  the  objective  function  altered;  but, 
given  the  model  and  its  objective  function,  experimentation  does  end. 

Simulation  models  have  no  such  built-in  assurances.  It  may  be  possible 
to  identify  improvements,  but  one  can  never  be  assured  that  some  small  change 
will  not  result  in  a  major  improvement.  Progress  is  being  made  in  the  devel¬ 
opment  of  techniques  for  the  systematic  exploration  of  simulation  models, 
e.g.,  SimOptimization. 5  However,  currently,  a  systematic  means  to  locate 
improvements  is  still  unfound. 


CONCLUSIONS 

Simulation  is  a  powerful  technique  that  has  been  used  in  the  past  and  will 
be  used  in  the  future  to  solve  important  problems.  Simulation  is,  however,  not 
a  cure-all  or  a  panacea.  It  is  a  technique  of  last  resort.  A  perfectly  valid  goal 
of  simulation  may  be  the  organization  of  ideas  and  knowledge  of  a  problem  to 
the  point  where  other  techniques  may  be  applied. 

It  is  not  uncommon  for  simulation- model  builders  to  discover  that  they 
can  predict  the  results  of  runs  using  their  models  with  relatively  simple 
analytical  procedures. 
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If  this  is  true,  then  simulation  analysts  should  study  optimizing  techniques 
thoroughly,  both  to  avoid  attempting  simulation  of  analytically  solvable  prob¬ 
lems  and  to  direct  their  thinking  toward  the  kinds  of  solutions  that  may  be 
most  useful  and  productive. 

It  is  concluded  that  simulation  models  should  not  attempt  too  much  in  one 
leap.  If  massive  breakthroughs  are  attempted  in  one  step,  the  results  will 
likely  be  unverifiable.  We  may  be  unable  to  distinguish  the  results  that  are 
truly  new  information  about  the  object  process  from  those  that  are  aberrations 
of  the  model. 

Finally,  we,  as  computer  professionals  and  those  to  whom  simulation 
problems  ultimately  come,  should  be  most  careful  in  explaining  to  the  customer 
what  he  may  reasonably  expect  from  simulation. 
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